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ABSTRACT 

Directed primarily at teachers in bilingual progreuns, 
this report suggests ways in which they may assist evaluators in 
improving the quality of evaluation, particularly regarding 
descriptive information about the bilingual classroom. Six principles 
of classroom description are examined: (1) program model, type, and 
design; (2) student characteristics; (3) instructional methods; (4) 
teacher and student language use patterns; (5) functional language 
ability; and (6) development of language skills. Teachers are asked 
to respond to an example that highlights one or more aspects of each 
principle; emphasis is on practical, immediate steps that the 
classroom teacher can take to improve the quality of evaluation. It 
is noted that some of the information about the classroom may be 
relatively inaccessible to anyone but the teacher since It reflects 
cumulative insights gleaned from repeated encounters with students on 
a daily basis. Contains approximately 60 references. (LB) 
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In this work Dr, Andrew D. Cohen cbrilies the types of evaluation 
applied to bilingual educatron programs and suggests ways in which 
teachers can assist evaiuators in order to ensure accurate assessment of 
such programs. The discusstons of general principles, specific class- 
room situations and practical recommendations for teadiers are also 
intended to enhance administrators' and evaiuators' understanding of 
teachers' contributions to the evaluation process. 

Dr. Andrew D. Cohi»n is sentcw lecturer at the School of Education 
and director of the Centre for Applied Linguistics, Hebrew Univenity 
of Jerusalem, He is chairman of the Israeli Association for Applied 
Linguistics and associate chairman of the TESOL Research Committee; 
currently he is also a visiting professor in the TESOL program at the 
University of California, Los Angeles. His publications include A Soiio- 
ImgmsUc Appromh lo Bilmgual EJmaiwn (Newbury Hoi^, 1975), 
Evalualmg Evahalion, with M. Bruck and F.V. Rodrfguez-Bromi (Center 
for Applied Linguistics, Arlingloa Va. 1979), and Testing Langmgi 
Abilify in iU( Cloi^mwm (Newbury House, 1980). 

One of the activities of the National Clearinghouse for Bilingual 
Education is to publish <k)cuments addressing the specific information 
needs of the bilingual education community. We are proid to add ths 
distioguished title to our growing list of publications. Subsequent 
Gearinghouse products will similarly seek to contribute information 
and knowledge that can assist in the education of minority culture and 
language groups in the United States. 
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Introduction 



As bilingual education continues to be a prominent force on the NcMlh 
American educational scene, evaluation of bilingual prc^ams b be* 
coming inaeasin^y refined. Ptacticirg and proq;>ectfve evaluators are 
now able to obtain exteiwive if not exhai^tive de$cri{:rtk>ns of what it 
is they may or should evaluate (see, for example, Bissell, 1979; Burry, 
1979). 

There are two l^ic types of evaluation, formative and summative 
(Scriven, 1967), Fimmtlivt emttuUum involves data gathering to help im- 
prove the program under development • Such evaluation is intended to 
provide feedback to tea«:hers at regular intervals, lhi» allowing for 
midcourse corrections if certain students are not progresslr^ ade- 
quately under the given instructional approach* Sttfrnm^livf epnlualwii is 
concerned with producing terminal judgments about overall program 
worth. Such evajuation deals with the generalized, long-range effects 
of the project. It usu«ily takes the form of standardized tests or 
project-made tests of general skills (those not taught only to students 
in the project)* given on a pretest-posttest basis at the start and finish 
of each school year. 

Program evaluators are often brought in from outside to increase 
the objectivity of the evaluation, and they are often requested to focus 
on summative evaluation. If the evaluators are external to the pro- 
gram, they may, consequently, have limited knowledge about the 
range of special factors pertaining to the particular bilingual program. 
Hence, it seems reasonable for teachers to think more ?bout steps they 
can take to ensure accurate evaluation of their prc^ams. The teacher 
can play an invaluable role both in guiding evaluatcHS in the desi^ of 
evaluation by identifying key variables lo consider and in l^lping to 
interpret the results once the data have been analyzed. 



It imy be arg^ that if b ncrt a teacher's bigness to get involved 
in evaliuiion. it was (oimd In one study {Cch&\ and Rdl 1979), for 
examf^. that teadm h«l diffkufttes fiflierfaig to strii^aot gukkilnes 
for colkcting oral (^natural'^ data from children in their d^ssrooms. 
for transcribing these d^a* and scoring them. On the other hand 
teachers' as^imc^ in desiring evduatim am! in interpreting re- 
sults may be crucial if meaningful evalu^icm is to take place* Te^ers 
can in evaluation by descriMng wh^ they alre^y are aware of 
and by turning tl^ir awareness aiKi powers of observation to class- 
room |^;KMTiena they did not previoudy attend to. 

WitlK7Ut some assistance from fnoject staff ^and teachers seem to 
be the best source*— evaluators may draw inappropriate conclusions 
abojit the nature of bilingual education classfX>oms, conclusions that 
lead to statements which misrepresent the reality of these classrooms. 
A recent Urge-scale evaluation of tnlingual sdwoling (Americiui 
institutes for Researdi, 1977), tw example, came under criticism for 
failing to pay adequate attention to factors characterizing the actual 
iMlingud education classroom, siKh as distinguishing student and pro- 
gram charaderi^ics. 

What are the advantages of accurate evaluation? Why should the 
tether be motivated to participate in evaluation efforts, however 
minimaiiyT A recent convocation of experts (Bbsell, 1979) produced 
rtie foUowing 1^ of potentid l^nef its of evaluation reports: 

1. Identifying project components that are highly successful 

Z Describing a project to parents of participants and nonpartici- 
pants to encourage interest in it 

3. Informing dectekvi makers at^out the rationale for and the ber^fits 
of the project 

4. Conveying information about the project to other locations and 
institutbns of higher ^ucation 

5. [providing a morale booster for teachers and administrative staff, 
who see the evaluation report as testimony to their efforts in 
making the pro^ct work. 

Obviously, there b also the real possibility that evaluation will 
point up certain weaknesses, and the concerned teacher should wel- 
come such information along with the suggested modifications that 
mi^t 5^em from %ud\ findings. 

This mcmograf^ is directed primarily at teache?^ in bilingual pro- 
grams. Its intent is to suggest ways in which they may assist evaluators 
to improve the quality €>f evaluation. The monograph will mostly 
speak to those aspects of evalui^km that rdate directly to teachers and 
their work-*namely, descriptive information about the bilingual class- 
room. The monc^aph is also cfirected to administrators and evalua- 
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ttm in blingu^ ediK^km to enhtfKc Hiek mufentaiubig <rf how 
teadiers cm contribute to th; cvaliurtive frocess. The fal^N>rtinoe of 
moving beyond proiect documents fouml in ^kimfaifetratfve calces to 
^uid cbssrooms th^ the documents may describe proffm dw- 
acteristics which do mrt reflect tfte actiul state of ^airs in t}% cla^- 
room as the teacher knows it to be. 

We Will fociJ» our attention m some erf the priiK^iIes of ^sioc^n 
descriptim— principles that are intemfed to be parikiilariy rrievai4 to 
evaluatifH) of classroom aspects <rf Irfiingual ediKuAkm pwgf asm. The 
principles selected for dtecusston Mt (1) prc^am mo<M, type, and cfe- 
sign; (2) student charact^ics; (3) instructicHial methods; (4) teadier 
and student language use patterns; (5) fw tional language ^lity; and 
(6) development of language skills. 

First, the principle b presented. Second, an examine that h^lights 
one more aspects of this principle is described Mefly. Thiid. you 
are asked to respond to the example through an ex^dse. These exer- 
cises vary in nature according to the priiKi|4e involved. Exercfecs 1 
and 3 call for interj^etatim of results, while exercto 4 c^ both for 
interpretation and for recommemialions as to chmgfaig teachbig 
patterns. Exercise 2 calls for si^esting a meaning groiq>ing of 
student test scores for purposes of data tfialysk, exercise 5 calk for 
suggesting an alternate a^miach to langt^ asse»ment, exer- 
cise 6 calls for suggestions for assessing the re^lii^ process and for in- 
vest^ating a poor score on a stamlardized re^ng test. Details are 
often omitted from the ckscriptiom the situ^^km in orifer to aUow 
opportunity for more creative thou^t in the exi^isc phase. In each 
exercise you are asked to play the role of the tead^r in the particular 
situation. After each exercise, a dtecussim will be provided For tl^e 
sake of simplicity. En^tsh is consiikred the dominant socMial 
language in all the situations f^esented, and the nondominant lan- 
guage is referred to as the "minwity language." 

The case-study part of thb paper does not concern itself with 
nondassroom variables such as parental involvement, nor Is any effort 
made to deal with the complete range of classrocm vari^tes in- 
fluencing the outcomes of bilingud program evaluation. Disomion of 
such variables has been saved for mention later in the monogr^. 
The main intent is to emphasize pr^ticd, immediate rteps that the 
cld^room teacher can take to improve the quality of evalu^kni. ft is 
my hop^ that a discmsion of ceritain princi|^e$ in evaluating l^in^ial 
classrooms may contribute to improving the evaluatton product. The 
selection of principles for discmsiwi (as well as the po»il^ In- 
herent in the selection) reflects to a large extent the autlwr's ex- 
periences as an evaluator in four different evaluatkm contexts— three 
of them small-scale and cw statewide. Thus, the moiK^a{4) will at 
times be offering a p^ticular point of view, which the reader may or 
may not agree with. The important thing is to consider the issues. 
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Principle 1 



Program Model, Type, and Design 



Sime thf ^HutuuLtt program model, lypi\ and design of bilingual 
nhtx^lmg itin have an important effect on evaluation results, the in- 
dividual a^pi^ fs should l>e i ar rf ully identilieil and described. The pro- 
gram model refers to the basic broad classes t)f bilingual pri>grams 
(eg . transilional maintenance, enrichment, restorative). The program 
type refers to tht' specific characteristics of the particular m<xiel (e g , a 
maintename program that promotes minority language maintenance 
in all subject areas v one that focuses on maintenance in selected 
areas). The pM>gram design refers lo characteristics of the particular 
program type (eg. within a full bilingual maintenance program, the 
first and senuvl languages are both intrcnJuced in early stages and 
emphasized equally later on) (See Trueba, l^7Q. ftu a complete dis- 
tussu>n of the distinctions between mixlel, type, and dt»sign.) 

Regiirdmg the program model, the evaluator would need to 
examine and indicate the following in the report: 

• h the pn^grain liansitional. maintenance, enrichment, or restora- 
tive? 

Regarc^ing the program type 

• How IS bilingual instruction actually implemented? Are the twc> 
languag . used concurrently as media of instruction? Are they 
u!»ed allernalely (e.g., mornings in one language and afh?rnoons in 
the c>ther: Mondays in one language and Tuesdays in another)? Is 
spt i iai instruction in English as a second language provided? 

• w much of the day— in actual classroom minutes or hours— i> 
•ser\'ed tor bilingual schcwiing? 
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• Is the progr^ni team^taughi with two or mure teachers? Are there 
one or more paid aides in the classroom? 

Regarding Ihe program design: 

• What are the rthntc and language backgrounds of the students? 
How are the stinients grouped in the classriMim {r.R,. homo- 
geneously or heten^eneously by native language)? 

• Are the students in a seIf<ontained classroom for bilingual in* 
struction or do they receive bilingual instruction In several class- 
rooms <as through team -teaching, departmentalized instruction, 
or puil-out programs)? 

• How many of the school's classrooms and which grade levels 
have bilingual jnstruition? How many srhtnils in the district are 
im iudfd in the bilingual program? 

Across mtidel type, and design: 

• Have there Iwn major changes in any of the above-mentioned 
factors over time (i r.. is the program stable or changing?)? 

EXAMPLE 



Suppose that your scht^il district has a bilingual program in three of Its 
schools. Two schools are entirely reserved ftn bilingual education 
grades K-4. while one schwl has a program in half of its classrooms 
grades K-4, You are a teacher at this third school. !ra( hing in the bilin- 
gual program at the fourth-gr^ level. Two-thirtis of your students 
are native speakers of the minority language; one-third are native 
speakers of English and memt>ers of the majority cuiture. Your entire 
day is devoted to bilingual schooling in a self-contained classroom. 
You have a paid paraprofessional aide in your classroom. You and 
your aide are both native speakers of the minority language. Your 
own English skills are as good as your minority language skills. Your 
aide is stronger in the minority language. 

The program has been in existence for five years. The text of the 
original project proposal stated that the program was intended to 
maintain and preserve knowledge of the minority language As of last 
year, the proposal began to emphasize transition from bilingual in- 
struction to English-only instruction, with the result that learners are 
now being prepared to make this shift as soon as they are ready. Three 
years of bilingual schooling is now profK>sed to be the maximum 
amount of time they need to develop skills considered adequate for 
exiting from the program. 

Evaluators have just tested altitudes of students in your class to- 
ward self, school and community, and found results that you had not 

^ * ^ rs 
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expciied. Your minority ianguige students, who have been in b^ii^iua! 
schooling for five years, con\e out lower on scales of self -«cceplarK:e, 
social maturity, school affiliation, achievement motivalioa and social 
security than students in lower grades who have been In the pro-am 
for fewer yeais. It is your firm belief that you are providing the stu* 
dents with an educational program suited to their neeck and desiref, 
but the students' attitudes according to the assessment seem to bdie 
this belief. 

EXERCSE 



The objective of thir. first exercise is to identify aspects of the pro- 
gram model that would affect student performance on such an atti- 
hidinal instrument. Drawing on the previous brief description and on 
c^her dimensions that may nut have been spelled out, list some factors 
that might have prcxJuced these apparently negative student attitudes 
toward self, sc hooK and community. 
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DISCISSION 



^ » *^«^ fa>tf>e program, you may well be abfe to help the evalua- 
lor intopret the firaiings. For example, it may be that sIikc ody half 
of your dassroom at each grade levd are designated for bilingual in- 
strudfoa these classrooms have been stigmatized by the shidents as 
bBir« remedid and therefore less desirable than the mainstream dass- 
•liu^' * may have an Inaeasing impact upon the stu- 

dents the longer they are in tl« program. Such a stigma could tarnish 
the students' altitudes toward themselves, toward school, and toward 
their status in the community. 

Abo, the program may well have started as a language main- 
tenance program to help instill in the students a sense of pride in their 
naHvelmguage and ethnic group. In fact, in the earlier grades the 
atmophere may still be one of language maintenance. However, it is 
possible that at all grade levels, or at least in the upper grades, the 
prog'am has evolved into a transitional one, with a premium being 
put cm gaining mastery of English and doing so quickly. Regardless of 
whether you or your colleagues are overtly playing up the English 
language and mainstream ethnicity at the expense of minority lan- 
guage and ethnicity, the students may well be getting this message. 
(See Trinciple 4: Teacher and Student Language Use Patterns," for 
more on how tpa'-hm may signal language preference.) 

Of course, a scries of other factors might have affected the re- 
sults. Perhaps the measure of atHhufcs dkl not yield an accurate assess- 
ment because the studenh: did not respond reliably (e.g., they did not 
understand wh. , desired, they were unwilling to cooperate, due 
to fatigue or wh r). Sometimes teachere are in a good position to 
evaluate this aspect, if they are present when the instrument is ad- 
ministered (but, ideaUy, not involved in the actual administration 
process). Another problem is associated with giving the same alH- 
tudlnal measure across grade levels. Students at lower levels may not 
be as capable of providing accurate answers as older studenls. Some- 
times there are different versions of the instrument for different 
levdb, but how equivalent are they? Issues like these are usually be- 
yond the so^ of the teacher's involvement in evaluation, but evalua- 
tors need to consider them. 

Assuming the instrument was obtaining reliable answers, then 
Aere arc stiU several other possible explanations for the results. One is 
the novelty effect. Enthtsiasm for any program may taper off after 
several years. Most of these fourth-grade students have been in the bi- 
lingual program for five years, and so much of the novelty in what 
they are doing has already worn off, and routine has set in. Second, 
perhaps societal forces toward assimilation are powerful enough to 
communicate to the students the importance of being in a mainstream 
program compared with a bilingual one, 

^ BEST copy AVAILABLE 



It is also important to ssk wither otl^r ^peds of the progmm 
might tiave an effect on student attitudes* examine, it was sUted in 
the cfesaiption ctf the situation that you teach in a ^if -ccuitained da»- 
room. In a study comparing bilingual prc^nura m self-cOTtairad dffs* 
rooms With those i^ing separ^ rooms for eadi iai^uage, studaits 
scoi^ bwest in EngtidK reading and in math Mdten ^dying in self- 
contained cla^rooms ar^ h^h^ in Engl^ reading, Span^ reeling, 
and nulh when studying in sqwate rooms (Qiicago Board of Educa* 
Ikm, 1977). These kir»k of findings simply play up the relevance of 
accurate program description in interpreting student adiievement in 
bilingual programs. 

It would also be interesting to look at the grouping of students in 
the classroom. If during a major portion of the day, the minorily lan- 
guage speakers are separated from Itie Er^ish makers, this could ad- 
versely iffect students' attitudes toward the bilingual fm^ram. It has 
been pointed out that grouping itself "comtihates a strategy of bilin- 
gualization . . . since much of lai^uage learning m achieved thrtntgh 
pupil interaction'' (Mackey, 197S, p. 13). Grouping cmild intply as^- 
ment to separate classrooms according to language dominance or to 
separate groups within ttie same ct^room. Both appro«K:ltes may 
have similar effects. 
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Principle 2 



Student Characteristics 



Accurafc characteri/aJion of students is not easy, but it is essential if 
the results of evaluation are to be meaningful. Current theories about 
bilingualism and cognitive development have helped provide more 
precise ways of describing student char^leristics than in the past. For 
example, ihv following variables shc^uld be considered. 

1. The student's ahi^lule proficiency in each of two languages, not 
just rfhihve proficiency {usually referretl to as Im^mige ihmtnimf). In 
other words, what seems to he the student s total grasp of vocabu- 
lary in both languages? Perhaps receptive reading vcKabulary is 
relatively modest in both languages (i.e,, with regard to a basic 
wonJ list). What is the child s control of the grammar of both lan- 
guages? For example, ihe student may appear to have control of 
most, if not all, of the grammatical structures in both languages 
that a native speaker t^ie same age would have. The concept of 
absolute proficiency is an important one. Sometimes statements 
of relative proficiency are misleading because they do not in- 
dicate limitations that students may have in certain areas {e,g,, in 
v(Kabulary) of holh languages. 

2 The student s level of cognitive functioning in the classroom. 
Theory now has it that bilingual children need to develop pro- 
ficiency in at least one of their languages above a basic ihreshold'' 
level in order to learn cognitive structures without difficulty 
(Cummins, 1978). Teachers may have son\e insights into the ease 
or difficulty with which particular students learn new concepts in 
class. They may be aware of particular students who have trouble 
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learning a makh concept when presented both in English and in 
the minwity language. Then the tejK:hers noti^ certain limita* 
tions in the akoluk prc^ciency of the students in the two Ian- 
gu2^, aral a reason for the concefrtual difficulty presents itself: 
the students have difficulty conceptualizing altogether. They 
need to develop laf^uage prttf tc^ncy m<^e fully (according to the 
theory), po'haps with an initial emph^is on the mother tongue, 
the mincmty language. A student with low language proficiency 
in both Unguages has been referred to as a "semilinguar rather 
than a 'l^iltt^uaL'* Semiiingualism implies limited develqpment in 
both langua^ of dimensions erf lar^uage proficiency strongly 
related to overall cc^itive and academic skills (Cummins, 1979). 
Furthermore, semiiingualism does mrf necessarily imply lower- 
class status (Skutnabb-Kangas and Toukomaa, 1979). 

We would want to add to proficiency and cognitive functioning 
more traditional variables such as the following: 

i, identification of the student's ethnic ^oup, number of years in 
the United States, language use outside of school and socio- 
economk: status. 

4. The student's educational history, particularly the years the child 
h^ been in the bilingual fm)gram. For example, one student may 
have attemied an Engl^-cmly kindergarten and entered the bi- 
lingual program in gr^e 1; another student may have been en- 
rolled in a mcmolingual first-language prc^ram in the native 
country for three years before coming to the United Slates and 
entering the bilingual program in grade 3. 

EXAMPLE 



Suppose you are a first-grade teacher in a bilingual program that was 
set up to ijKlude two-thinfe bilingual children and one-third mono- 
lingual Engikh speakers, half erf whom are ethnically members of the 
minwity group. Five of your students just arrived from another 
country, where \hey attended kimfcrgarten in the langu^e of that 
country. Another five were transferred into your class from a mono- 
lingual English kliHiergarten, largely becaise ;hey are native speakers 
of the miiwily language and it h spoken in their homes. The re- 
maining ten minority language speakers come from families living at 
low socioeconomic levels, where the minority language b spoken 
almost exclu^vely and where there are few educational stimuli en- 
couraging envelopment of native language proficiency among tl^ 
children (e^^ecialiy with regard to reding and writing). These chil- 
dren have also not developed extensive proficiency in English be- 
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cause <^ limited contact with tibe language and because tf^^re is no 
need to it at home. All Hie numdingual English speakets partid- 
pated fa) the bilingual kfaKkrgnlen but f^ked up relatiwly littie 
minority language {^trfic^ncy becai^ it was not essential. All acH- 
viH^ were conducted in EngU^ 9S well. 

An evaluator comes into your ck^sroom, tests the children as a 
sins^ group^ and rq>orb grcH^ n^an scores in reading in Ei^^ and 
in the minority language. Hie reading scores are poor Ixith in EngUdi 
and in the minority lai^uage. The evaluator writes a rqx»t that is 
critical frf the bilingual education ]»t7gram. 

EXERQSE 



You can see that it may be meaningless to lump all the scores tt^ther 
for purposes of analysts. As an example, write down your ideas for 
grouping students for porp(»es of analyzing their semes. After that, 
suggest ways in which the program itself may have contributed to 
limited reading achievement, under the assumption that the program 
itself can be improved. 
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IMSCUSSION 



SbKe ti^ evaluaHon is supposedly measuring tl^ effects bilingual 
schooling, it wpuld seem crucial to citify students according to the 
amount of bilin^ai schooling they have receiv.'d. If there h a sub- 
group of stiHiente in your classroom whu attended br^ishH^iy 
kfiK^g^tea it would seem important to loo^ at tl^ir results s^* 
rately from thc^ of students who attended a bilingual kindetgarten. 
Aho, if there are stucknts who h^ a native langu^e ktn<krgarten in 
ttor country erf origin, this group shcnjid be looked at separately. 

Then, assumii^ some sort erf bilingt^l language proficiency test 
was given to the students prior to this bilingual reading test, it is im- 
portant to pay attention to the scwe each student received in each lan- 
guage, ideally, you will be able to say in grcKs terms whether a 
student's perfcmnance is poor in both languages, poor in one lar^age 
and fair in the other, fair in one and good in the other, and so forth. If a 
given chiW performed pcx)r!y in both languages, this child may be 
more of a semilingual than a bilingual, ^ described previomly. 

The semilingual, tlwn, is a studrol who wouki l^nefit from a 
strong native language reinforcement prc^ram before second lan- 
guage skills can be effectively developed {Cummins, 1979). If such 
students have been put into a bilingual reading program, the addition 
erf extra demamfe in tlw seccmd language couW explain poor perfor- 
mance on part. In other words, their limited native language pro- 
f kiency may have kept them from reachiitg a basic threshold of cogni- 
tive functioning ^he time that they were requested to start develop- 
ing language ^! . id concepts in the second language. You arid 
others in the pri.b . n may have thought that they had mastered cog- 
nitive slruchjres in both languages, given their superficial fluency in 
the secomJ language (i.e., appropriate accent, intonation, and common 
expressions such as greetings, requests, exclamations, insults, and so 
forth). But thfe apparent fluency may actually have represented a 
fac^e masking the child's lack of mastery of basic concepts (see 
Cummins, 1978). 

Another relevant question is the ethnicity of the students. Looking, 
for example, at the minority language reading scores of the English 
monolir^ual students (five from the majority group and five from the 
minority group), it may be that the majority -group students perform 
somewhat better than the minority students of similar cognitive 
ability. Why woukl this be? Perhaps the minority students are less 
keen altitudinally about identifying with their ethnic group through 
reading in that language than are l\w majority students who have less 
emotional investment in the undertaking. Such results have actually 
been found in certain communities (e^., in Rosemead California; see 
Cohen, 1975a). In other words, the English-monolingual minonfy 



ERIC 



shkienls may come ml pooler cm a concreK rfcmnance impure in 
the minoiity language, si^h «s reading* If so, the expbnatkMi could be 
a con»:iou$ or unconsdoi^ ambivalence about identification with 
their language of origin* 
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Principle 3 



Instructional Methods 



As teachers know only too well from daily experience in the class- 
room, the instructional method in a bilingual program is not simply 
'Ijilingual education/' altlK>ugh it nruikes evaluation e^ier to look at it 
that way. In reality, the smxess of a prc^am may well depend on the 
nature of the spftifu appro^h to oral skilk to reading, and to subfect 
matter With respect to an oral language prc^ram, for example, the 
evaluator needs to know the following: 

1 . Is there formal oral language instruction in the second language? If 
so, to what extent do the instructional methods emphasize 
meaningful communication as opposed to focusing strictly cm 
form? For example, memory of dialogs and pattern practice drills 
may emphasize form extensively and perhaps exclusively, 
whereas practicing ullerarK:es in meaningful contexts and en- 
gaging in communicative exercises may emph^ize natural lan- 
guage more. 

2. Are there teacher guidelines for the program? For example. In 
some programs teachers use ready-made le^on plans prepared 
commercially or by the local district. Such lesson plans take the 
form of actual scripts that the teachers are to read. If there are 
guidelines, it is useful to know how carefully teacher? adhere to 
them. 

With respect to reading, the evaluator needs to know the 
following: 

1. Which language did ihe students begin reading in? Their native 
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Ungujq^eTThe secof^ ianguage? Or dkl they start reading in t>oth 
languages simultaneously? 

2. Is there a particular reading method or con^ bination of methods 
being used for teaching reading in each language (e^., the 
phonemic approach, the linguistic -phcmemic approach, the lan- 
guage experience approach; see Aukerman, 197 1 )? 

3. Are the materials for teaching reading in both languages equally 
good? Sometimes materials in one language (usually English) luve 
been field tested more extensively, ami only the most interesting 
and challenging stories and exercises have been retained in the 
final commercial version. Field testing also helps to establish 
whether the particular approach to reading seems best for the 
majority of students in the given lar^uage. Sometimes one read- 
ing series includes a better system for giving students feedback 
about their progress —either through step-by -step programmed 
activities or through meaningful tests at the end of each unit. 
Some materials are tetter than others at recycling previous 
material to reinforce vocabulary and structures presented in the 
earlier material. 

4. Assuming all students are using the same method, is the material 
sequenced tlie same for all of them? 

5. Do students who are using the same readers progress at the same 
pace or at different paces? Do they progress by group or indivi- 
dually, as through the contract approach, whereby each student 
makes a contract with the teacher as to how much he or she will 
read? 

6. Are students grouped by ability, by interest, or homogeneously 
according tu language dominance? Is there total -class instruction? 

7. Is scheduling flexible? Are students in nongraded classm-*- . 

6, What IS the ratio of teachers and paraprofessionals to children? 
What is the language proficiency of the instructors in each of their 
languages? 

9. What efforts have been made in the classroom to promote bi- 
literacy (e.g., bulletin boards, store corners, reading centers, signs, 
school/classroom post office, newspapers, magazines, Ixxiks)? 

EXAMPLE 



You are the minority language teacher in a team -teaching situation in 
a bilingual third-grade classroom. You have been teaching only three 
years. Your colleague who concentrates on English language instruc- 
tion has had fifteen years of experience teaching English language 
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skills. Hie method for teaching oral skilk in Ei^iish as a seccmd lan- 
guage b similar to that (or teaching the minority lan^age as a second 
language to Englbh -dominant children. In both c^es, students receive 
pattern practice drilb consisting of senterKes that ti^ are asked to 
repeat individually and in groups and then modify acccNxbng to the 
model. 

Because of pressure at the distrkt level not to delay En^ish reading 
for a year or two, all dnldren began reading re^iness in bc^h En^ish 
ami in the minority language in kindergarten and started receivii^ 
reading instruction simuitaneoudy in En^ish ami in the mim»'ity Ian* 
guage in first grade. Whereas the En^ish reding program uHliz^ by 
the district is an eclectic one which has t>een in use for a number at 
years (drawing upon the best from a variety of methoifeX the minority 
language program is a recent effort, consisting of materials that have 
not been field tested to any extent and which the students are not too 
enthitti^ttc about. They complain ttiat the stories are sometimes too 
difficult and not very interesting. H^y find that their En^ish reacfen 
are nrK>re colorful and fun to read. Furthermore, where^ students are 
given freedom to adjiHit tl^ir English reading prc^am to their in- 
dividual needs, the minority lar^iuage pr<^am is more tightly struc- 
tured. All students, whether &^tsh dominant or minority lai^age 
dominant, are given the same reading materials and grouped accord- 
ing to ability. 

An outside evaiuator has just tested your students and \m found 
that the students who are suf^Kisedly mincH'ity language domii>ant 
actually score surprisingly better in Englsh reading than in native 
langu^e reading although these English reading scores re^h only a 
modest percentile on the national norms. Their En^ish oral language 
scores, however, are ncrf as high as their English reading scores. 

EXERCSE 



The school administrators are most upset about these results and look 
to you to help interpret them. What would be some of the possible 
points that you would offer in your interpretation? 
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DISCUSSION 



EvaiuatcHS who anaiy/e test scores apart from the classroom context 
ios« a good deal of relevant information. The choice of oral language 
programs couid help explain limited progress in oral second language 
skills. Having students repeat or imitate patterns (words, (Erases, or 
sentences) presented by a teaclwr, then having pupils alter these 
patterns or prixluce new patterns, may not stimulate student oral Ian* 
guage growth very much. This, in fact, is what Ramfrez and Strom- 
quist (1979) found in a recent study. Legaretta (1979) also looked at 
English as a sei€>nd language (ESL) training in bcrfh traditional pro- 
grams arni liiltnguai programs She found that although the audio- 
lingual approach to ESL appeared to aid comprehension of basic 
English vocabulary, it did not enhance competence in a communtca- 
tfve situation [^ulay and Burt (1975) give theoretical and empirical 
grounds for why an ESL curriculum based on imitation, repetition, ami 
memorization of 5»entence patterns does not enhance performance in 
communicative situations. They suggest that providing a significant 
amiHjnt of class time for natural communicative situations would 
allow learners to better use their "creative construction" abilities (Le., 
their abilities to acquire language on their own). 

Furthermore, a teacher's experience in teaching reading can help 
increase the students' reading scores, regardless of the language— and 
in the bilingual movement, it is usually the English reading teacher 
who has had the opportunity to accrue more years of experierKe, as in 
this vignette. By the same token, publishers have had a good deal 
more experience publishing successful English reading curricula than 
reading curricula in other iant^ages. More recently, minority lan- 
guage reading materials have improved in quality; nonetheless, the 
second language market is not characterized by the kind of intense 
competition that the English reading market faces. All this simply 
means that English reading metlwlology has an edge, and that it is im- 
piHtanl to dtxument the extent to which the minority language read- 
ing program is equivalent to its English language counterpart. 

It is also worthwhile to note any major changes in the minority 
language reading program during the year or during the previous 
several years In this vignette, it was pointed out that the minority Ian* 
guage reading prcjgram was more tightly structured than the English 
reading program, and not tested very much. There have not been 
many tests to choose fuym for assessing reading in the minority lan- 
guage, and if sue h tests exist they have nut been normed as have their 
English language ctumterparts. In other words, it has not been al- 
together clear what a certain score on such a lest means regarding 
reading level, given a student's particular grade level. 

The depressed nature of the English reading scores among the 
minority group students, even given thai some are monolingual 
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Engli^ leakers, suggests that some of these students may well be 
semilinguals (see Pru1c^^fe 2)— i-e^ students who would benefit from a 
native lai^u^^ reading program for a year or more, until their cog- 
nitive skills are better developed. Tlie simultaneous introductfon of 
reading in the minority language and in English probably creates con- 
flicting signals for them And in tlwt the Englhih reading materials 
have the instructional advantages mentioned previous^ly, the students 
are also probably nwe motivated to read Ihcm. Of course, this 
vignette does not descrit^ the degr^ of minority language reading 
readiness provided in the home, but let us assume that it is limited in 
that the parents had limited schmiing themselves and have limited 
means for purchasing Inxjks and other educational stimuli for the 
home. 



Principle 4 

Teacher and Student Language Use Patterns 



The leathers thoke oi langua^f for ilassn^vn fnanagemrnt, for dis- 
cussing cimlcnt and for chattinK with lolleaKuini durinx breaks may 
well Iransmil attituties ahiut the rwo langiMKC?^ to the students. The 
leather's language use may, for example, mark one language (usually 
the mini>rity language) as less acieptahle or perhaps even inferior. It is 
for this reasim that it is advisable to audio- or even videotape lan- 
guage use patterns Suth external objective measures help teachers 
base their self -assessments upon ai.\ual behavior rather than personal 
impressions. 

The evaluator needs to know whether the teacher uses one lan- 
guage only for various ptntions of the day. or if a particular teacher 
uses only one language. If, for example, the teacher moves back and 
forth between languages, is this switching patterned and purpt>vHul 
{as in the concurrent appr-oath; see Jacolnon, \97o) or unsystematic? 
Switching from one language to amither -even carefully planned 
switching^may promote interference between the two languages 
(Cohen, I97Sb. chapter 8), particularly interference from English 
when speaking the minority language, since the minority language is 
usually the more susceptible to erosion in a sixiety where another Ian- 
Kuage is dominant. In other words, this type uf er4>sion of Iht native 
language, or backlash interference " as Jakolmvits ( 1P7U) has called it, 
takiH* place in the Spanish of native Spanish speakers living in an 
English-sptMking country, in the English uf English speakers living in a ^ 
Hebrew-speaking country, and so forth 

To understarul language use patterns in the classroom, it is impin- 
tant to describe not only the language use patterns of the teacher, but 



also the UngiiaKe use patterns of Jhe students. Student languane use 
paHems may not conform to those trf the teacher. allhouRh they are 
often prompted by the teather's patterns. The language that shidents 
use for negotiating or managing activities among themselves, for re- 
s{K}nding to the teacher, and for thinking to themselves while reading 
or while trying to solve problems may be somewhat conditioned by 
the teacher's language use behavior However, students' language use 
patterns may also conform to their own patterns of sixial interai tion 
in the classrinim 

If the prt>gram is two way in that monolingual English speakers are 
learning through the mintirity language while minority language 
speakers are learning through English, the evaluator may wish to 
know whether there is genuine two-way bilingual language use. or 
whether English language use prevails among the monolingual English 
s|;^akers. Monolingual English-speaking minority students may be 
particularly resistant to using the minority language. It may be that 
they are monolingual English speakers haaust they rejected using the 
minority language early on. Perhaps they consciously or uncon- 
sciously sensed negative attitudes in the community at large toward 
the minority language and felt it better to ftKus on the majority- 
group language Perhaf>s their older siblings or parents had already 
made that attitudinal det ision before they were horn and transmitteil 
it to them, There may also be other re^^uns for English-speaking 
minority students to resist using the minority language 

EXAMPLE 



Evaluators from the federal government come to your fifth -grade 
classroom with stopwatches and ret *)rd the number of minutes of in- 
struction that take place during two hours in both languages They are 
concerned about whether each language is getting "equal treatment " 
As it turns out. the vimIofs are present mostly for small -group reading 
in the minority language and for mathematics which is being taught 
concurrently both in English and in the minority language. 

In this concurrent methtni you have decided to use either English 
or the minority language for one of four reasons; 

• As a conscious classr<Him management strategy (e.g.. to reinforce 
a concept, to review, to get attentiim. or to discipline) 

• Jo be consistent with the t uruculum (e.g.. to talk about a specific 
domain or topi* ) 

• lo help student?*" language development (e.g., to enrich the stu- 
dents' V(Kabuljries. to improve their ability to sjy ihe tame thing 
in two languages) 

• To improve intcrperst^nal relations 
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YcHj have chained thcwr oper^ing principles from Jacc^^son (1979). 
D^pite your establbhed set of {^rating i^incipies, you make a con- 
scious effort to i£9e the languages in a balanc^ way, as does your 
paraprofe^kHial aide because of the presence of the evaluators. You 
tend to addre^ the students in what you consider their dominant lan- 
guage. 

Although you are a native speaker of the minority language, when 
you take your break, you speak in English with your aide and with 
other colleagues both in class and in the corridor on the way to the 
teachers' room. In class, you are mindful of the languages that you and 
your aide speak. To a certain extent, you are also mindful of the lan- 
guages that the students use with you ami among each other* In 
iKtuality, they tend to prefer to speak English in most situatk^r^, even 
those students who arc supposedly dominant in the minority 
language. You really don't know which language they think in when 
processing mathematical information, science problems, or informa- 
tion in other fields. 

Your program is, in theory, a two-way program, in that mono- 
iinguaJ English speakers (both from the minority and from the 
majority groups) and minority language speakers are receiving school- 
ing in and through the minority language as well as through English. In 
reality, you have not paid too much attention to use of the minority 
language by English-dominant students, since their learning of that 
language has been viewed as less important for survival in the society. 
The principle target group has been those students who are dominant 
in the minority language. 

In their stopwatch proc«lures, the federal evaluators concentrate 
mostly on te^rher talk because it constitutes almost 80 percent of the 
classroom talk that b audible to them. They find that you tend to use 
the minority language more than English. They also find that you use 
the minority language primarily for classroom management activities, 
particularly for disciplinary purposes, and English to discuss content- 
not only in math lessons but in minority language reding lessons as 
well. 

The visit of these outside evaluators perks your own interest in the 
language use patterns of children in your class, so you decide to do 
several things. You place a microphone or two around the classroom 
and tape the language that bilingual minority students use among 
themselves. Interestingly, you find that they tend to use the mirwrihf 
language for management among themselves and English for discuss- 
ing content, such as for solving math problems. You also ^k students 
to keep logs of the language that they use to think in at various Inter- 
vals during several different lessons. Most of the students report that 
they are trying to think in English even if the formal language of the 
class is the minority language. 
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EXERQSE 



Given tl^ language use paKems desciit^ atn^ve, how would you 
chan^terize the su(^)ort that is beii% given to the miiXKity language 
and why? What do you think ihe relative effect of these patterns will 
be on minority language maintenance as opposed to transition to 
English language use? Assuming that you as a teacl^r desire more 
minority language me in the classroom, which patterns might it be 
desirable to change? How effective do you think such changes will be? 



DISCUSSION 



CounHng minutes of tnstrt^ion in e^h language is a questionable 
mear^ of determining whelher there is "equal treatment'' of both lan- 
guages, it is not so much the amount of time that each language b 
used, but what the functions of that use are. You may spend more 
minutes in the minority language, for example, but still signal through 
particular patterns that it is better to use English for "important" tasks 
such as solving problems, while the minority language is OK for 
disciplining students. You may also signal that it is more fun to use 
English when relaxing during a break. 

You shouki come out of this exercise with the feeling that ttie pro- 
gram just described emphasizes transition to English language use. 
Teacher patterns in the class are signaling that English language use is 
more prestigious, regardless of the teacher s efforts to make a show of 
using the minority language. If the teacher is interestetl in promoting 
more extensive minority language use. there are several possible ap- 
proaches: 

r Switch from a conci "«t approach to r -ye that devotes bnger 
periinis (even entire days) to minority language use exclusively. 
There is some evidence that alternate vse of the two languages— 
i.e., one language as the medium in the morning and the otiier 
language in the afterncx^n, without repetition of the same con- 
tent—may be more effective than certain concurrent approaches 
in prixiucing better gains in communicative competence in both 
languages, as well as in oral comprehension in English (Legarreta. 
1979). In all fairness, the concurrent approach studied le- ded to 
encourage predominant use of English (72 percent of l-<e time). 
Perhaps the concurrent model advocated by Jacobson (1979) 
ensures a more balanced use of the languages. But Dulay and Burt 
(1978) give theoretical supjn^rt to the opinion that students tune 
out the language that they do not understand and wait for the one 
that they do understand. 

2. Avoid marking the minority language as less prestigious by 
relegating its use in class to remedial or disciplirtary functions. The 
stuiients need to feel that the language has application to a variety 
of functions, 

J. be careful to use the minority language as much as possible out- 
side of formal instructional pericxK such as during breaks. In 
other words, give the students a live mcxlel of how the language 
can l>e used comfortably in less formal environments. 

4. Provide the students with frequent examples of how you "think 
out loud" in the minority language It may be that the students 
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tack the minority language terms for thinking through and ulti- 
mately solving problems. 

5. Provide activities for stimulating monolingual English speakers to 
use the mincMity language in the classroom, if this te a program 
goal. One suih activity is to engage these students in hnmenion in- 
structional activities wherein they are homogeneously ^ouped 
with other mondingual English s{^akers for certain sessions in 
the minority language so that they need not feel they are 
competing with native speakers of the language (see Cohen artd 
Swain, 1976). At other times, the ^diticipation of minority lan- 
guage speakers may expressly encourage the participation of the 
nornative speakers as well. 

In ;eaiity. bilingual classroom prc^rams may have some effect on 
language use patterns. bo\h at school and at home, e.g., in fostering 
continued use of the minority language beyond what that use would 
be without the bilingual program (see, for example, Cohen, 1975b, 
chapter 9). But ultimately societal forces may determine language use 
patterns; especially by the fifth-grade level deKribed in this vignette, 
those societal forces may l>e more influential than any that can be 
generated within the classroom. These societal forces include, first and 
foremost, the media— especially television, as well as newspapers and 
magazines. Also, the social groups which the student comes in con- 
tact with and participates in can t>e very influential, e^,, the local peer 
group, Boy Scouts, athletic teams. People on the street, merchants in 
stores, and others can also have a profound influence upon the lan- 
guage minority child, who most likely does not want to appear dif- 
ferent. Unless the society provides for i eguiar use of the minority lan- 
guage for certain social functions, it will cease to l>e used and the 
majority language will take its place (see Fishman, 1972, chapter 6, 
"Societal Bilingualism: Statile and Transitional"), 
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Principle 5 



Functional Language Ability 



Language researchers arc realizing that there is more to achieving lan- 
guage proficiency than mastering grammatical inflections, preposi- 
tions, negation, and interrogative forms. Assessment of speaking skills 
in bilingual programs has traditionally consisted of a tally of such 
errors (see Cohen, 1975b, chapter 8; 1950a). Thus, the focus has be^n 
largely on deviance or on what the student does not seem to have 
mastereci A complemrr ; and perhaps rr ♦^e productive approach 
is to assess what the student c an do with the language, particularly the 
ability to function successfully in it. In this functu>nal approach the 
emphasis is on the students command of communicative functions, 
i.e , the ability to perform certain speech acts in the classroom, to com- 
prehend what they mean when others perform them, and to com- 
prehend their meaning when they cnxur in reading texts. A "speech 
act" is the act of doing something in saying something. For example, 
native-speaking children know that they have a variety of ways to 
request something. So, in order to request that someone open a 
window, they may say, It's hot in here/' This declarative statement 
would then serve as a request (the speech act) 

The following is a tentative list of those speech acts identified by 
one researcher as most crucial to communii ation in the elementary 
school classroom environment: pleas, suggestions, requests, demands, 
orders, warnings, threat*;, promises, authorizations, and apologies 
(Walters, 1979). As it turns out, the speaker rarely uses the direct form 
of the speech act (e g., "1 request the eraser") but rather uses other 
strategies such as asking abi^ut ability ("Could you pass the eraserD, 
asking about location ("Where's the eraser?"), expressing a need (1 
need the eraser "), and so forth. 
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EXAMPLE 



An cessment of speaking skills has just been conducted at your bi- 
lingual schod, using a t^A with a series ot ^.«^ture-based items eltciting 
spoken resfHimes, each item assessing prof ktency with respect to one 
or more forms. For example, one item showed a broken juice bottle chi 
the floor with a disturbed youngster standing over it. The stinlent was 
asked, 'What happened to the bottler The item was testing for pro- 
ficiency in using the past tense <e.g.. The boy drapptii iV 'It fri// 
etc.). Other items tested for knowledge of various grammatical in- 
flections, prepositions, negation, and interrogative forms. The children 
from your second-grade class were tested individually in a separate 
room in both tJ^ minority language and in English If the student did 
not respond to an item, the investigator used prompting questions. 

Perusing your students' scores, you are surprised to find that these 
scores do not seem to give an accurate picture of the overall speaking 
abilities of your students as you have obtained it from hearing them 
converse in natural communicative situations on a day-to-day basis. 
You have a hunch that certain students have a good command of 
speech acts despite the conspicuous grammatical inaccuracies that they 
produce, and that other students may be good at grammar but limited 
in the ability to communicate functicmaity. 

EXEROSE 



Let us suppose that you feci the evaiuaior s concept ot language 
proficiency is tcxi restricted, Think of some approaches you might 
suggest to the evalualor in order to reveal a measure of your students' 
abilities to make requests or suggestions, promises, apologies, and so 
forth. The evatuator may be the final judge of what language instru- 
ments will be used in assessment, but it may be beneficial for you to 
provide input about areas worthy of investigation. Let us assume in 
this case that thanks to the suggestions you provide about how to 
measure success at certain speech acts, the evaluator alone— or with 
your assistance— decides to develop a test of this kind. You may have 
no formal training in the conbtrijction of language tests, but your daily 
teaching activities undoubtedly include a variety of techniques for 
eliciting meaningful, communicative language from your students, 
Desaibe these kinds of techniques as you would for the evaluator 
developing the test. 
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DISCUSSION 



Concern for assessing functiondi language ability of children in bi- 
lingual programs is growing (see Canaie and Swain, 1980). Omr con- 
cern IS to assess more than the student's knowledge about how the 
language works, specifically to assess the ability to demonstrate this 
knowledge in a meaningf uncommunicative situation. And within the 
area of performance testing, more and more emphasis is being placed 
on demonstrating a grasp of speech acts, such as requesting or 
apokigizing, Walters (1979) describes a testing situation in which an 
adult, possibly the teacher, uses a puppet to initiate a conversation 
with the student in four situations (supermarket lunch, outside at 
play, and outside selling cookies). This puppet functions as a "friend" 
of the child. In each conversation, the student is called upon to request 
something of a second puppet. Here is a portion of a conversation at 
the supermarket: 

Friend puppet: 'Wowf The lines are really long. I've got to get 
honoe by three or my mother will kill me, A^k this guy in front of 
us if we can go first/' 

Student: (Target request] 

Puppet #2: Sure, you only have two items, Co ahead." 

Of course, obtaining the data ts only half the task Determining 
how to analyie them is the other half. Too often, imaginatively 
collected da\d are submitted to a traditional count of grammatical 
errors. Assuming that the instrument is assessing a student's ability to 
make requests in a role-playing situation, Walters (1979) suggests that 
the student's score could be the number of different ways that the 
speaker requests things. This approach would require that there be 
enough role-playing situations to elicit the speaker s full repertoire of 
strategies. Another approach that Walters suggests is to assign each 
request strategy a value based on some prot>erty, such as politeness. 
Then the measure of communicative ability becomes the range in 
|7oiiieness that a speaker is capable of producing. 

Another apprc>ach to scoring has been designed for evaluating the 
ability to apologize in English (Cohen and Olshtain, 1980), First of all, 
the speech act of apologizing was broken down into four possible 
semantic formulas: expressing an apology (e.g<, 'Tm sorry"); acknowl- 
edging responsibility ( That was silly of me"), offering repair ("What 
can I do to helpT'); and promising forbearance ('It wont happen 
again"). The expression of an apology was further broken down into 
four subformuUs: expressing regret ('Tm really sorry"); requesting 
forgiveness ('Excuse me"); offering an afKjIogy CI apologize"); and 
offering an excuse ('1 forgot"). Then eight apology situations were 
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presented lo native speakers of English (the target bnguage) and to 
native speakers of Hebrew (the learners' native language). Their 
responses were used as a baseline for how natives apoic^ize in both 
langu^^. Tlien lexers of Enf^ish wen? asked to ^logize in the 
sanie situations. The data collected from native English speakers made 
it (possible to determine whether deviations from the typical apology 
in English was a result of interference from patterns of apoi<^ in the 
native langu^e {Hebrew, in this case) or simply a result of an in- 
adequate grasp erf English to enable such afK>logies to be expressed 
appropriately. The respcmses were scored according to how closely 
the student's apologies resembled those of native EnrJtsh speakers in 
each of the eight situations. The study was performed with college- 
age students, but it could easily be replicated with primary school 
children. 

Many individuals reading this monograph may not have the 
appropriate knowledge for judging the acceptability of present assess- 
ment practices and for recommending alternate strategies for assess- 
ing functional language. It is clear that this whole direction for lan- 
guage assessment is just twginning to develop. Although the theoreti- 
cal underpinnings have been available for years (Hymes, 1974; Ervin 
Tripp, 1972), little has been done lo put these principles in operation 
in terms of classrcx>m assessment Until such techniques have bi%.n 
more fully develof/ed. it will be difficult to say what the best means of 
data analysis is or what particular advantages such speech act analysis 
has over more traditional approaches. 
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Development o£ Language Skills 



The way that each of a student's two language's develops in a bilinKual 
priigram may be of interest, particularly in cases where the student is 
having difficulties. l}f I'thimtul here refers to the iruremental changes 
that nriay occur over time in each of a student's language skills. • j»eak- 
ing, listening, reading, and writing. Such information is not neces- 
sarily recoverable from evaluation of interim and end-uf-year 
achievement in a program. Teachers can provide some information, 
however anecdotal, e.g., a description of problems that the student, 
who is not very proficient in the native language, has when learning 
through both languages simultaneously. The ski!! that has received ex- 
tensive attention— perhaps because we seem to know the most about 
how to teach it --is reading. Perhaps teachers could write down com- 
ments that the students make or have the students explain the reading 
strategies that they are using to get meaning from the passage (see 
Hosenfeld. 1979). 

Even if evaluations include measures of achievement mute 
frequently than once a year, a certain amount of information is lost in 
the testing process because evaluators do not know what process 
students use to answer lest questions. I am suggesting here that the 
teacher can use the evaluator s standardized reading lest (and tests of 
other skills as well) as a vehicle for l>elter understanding where the 
poor reader experiences difficulty in reading. Perhaps the teachers 
could go over certain items on a test with the students, discussing 
how the students read t' fimulus materia^ md their reasoning be- 
hind choosing certain answers (see Cohen, 1980K chapter 3). It may 
be that a student answers certain test items wrong because of limited 
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reading ability, A discu^icHi fxssicm with the student cmiid help pin- 
point the scHirce of tl^ difficulty, if the teadier doesnit know it 
alne«iy (e^s problems decoding letters, limited vocabulary}. 

It k ako poK^ble that the processes wherri>y the sttdent arrives at 
answers to certain reading items are well based. i.e., based on sound 
reasoning, strong powers of inference, ami so forth. But the items 
themselves may have certain confusing prq?eriies. Of course, the 
main purpose of such an exercise is n<^ to critique the particular 
reading lest, but rather to determine what sf^ific problems the poor 
reader is having in trying to read— and specifically when trying to 
read material contained in reading tests. By talking with students 
about Itvetr answering strategies, it is possible to obtain some of this 
information 



EXAMPLE 



Ypu are a third -grade teacher in a bilingual program that has just 
begun a new schcx)! year. There is a group of ethnic minority children 
in your class who are having difficulty keeping up with the other 
children in reading in the minority language. As st^melimes happens in 
your class, some of the ht^l readers of the minority language are 
majority-group native English speakers. They have successfully trans- 
ferred English reading skills to reading in the minority language. Some 
of the minority students are also quite good readers in the minority 
language es|ieiially several who began their schooling in their 
country of origin. The children having difficulty reading started the 
program in kindergarten as limited- or non-English speakers. Their 
reading test score* at the end of second grade imiicate simply that they 
are having difficulty reading in the minority language. 



EXERaSE 



Think of ways that you could observe and assess the actual reading 
abilities aid difficulties that these students are having in the minority 
language Assuming that you could get copies of the reading tests 
given at the end of the previous year, how might you use these tests to 
belter understand how these students (individually and collectively) 
go about reading in the minority language? 
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DISCUSSION 



Evaluating are often mlercsted in reporting results kn large numbers 
of stuilents bt*cau!»e generalizations are easier to make from such re- 
sults. But one pri>blem with this approach is that certain kinds of in- 
formation are not us.ially collected. For example, if certain minority 
studentb who should be reading comfortably in their native language 
at the third grade level are not doing so, the evaluator's report may 
leave the reasons for this finding opvn for speculation. It. on the other 
hand, the teacher has some dixumentation about why such a 
phenomentin exists, it may be useful to explain the result as well as to 
suggest means for changing this situation. In all fairness, there are 
evaluators whi> engage in formative evaluation with an emphasis on 
the priKess whereby a student achievers an objective. But such evalua- 
tion IS time-consuming and it may not. in practice, yield the kinds of 
intormation it is theorelually supposed to yield Frc^m my own 
expera»nce. recording the attainment of reading objectivcni turns into a 
ftnm of biH»kkeeping. The current fcn us on reading strategies across 
objective's seems preferable. 

YiHi can begin by documenting the kinds of information about the 
reading program that are suggested under Principle 3. But even de- 
taiU alxuit the mstruitionai nuihtnls do no! necessarily provide any 
insighh coiui ming the stuilent's individual experience in reading. The 
student's individual experience is something th«il the teacher (or per- 
haps an evaluator) would probe with each student perstwally. It may 
be that a minority student is experiencing "backlash inlefferencc' 
when re»iding in his native language, ie.. his expectations about hoiv 
to get meaning from reading material are l>ased more c^n inflections, 
sentence strut lure, and vtKabulary of English than on those of his 
native language, l^learly much of the dcccxiing prou^ss is transferable 
from language to language, but tertain dilferencfs (such as word 
order) may prevail and may i ause diffit ulties. 

As a possible chetk tor whether batklash interterente is occurring, 
you tould ask sludents to express in hnglish what they have just read 
in their native language. It may l>e that their t:nglish language paia- 
phrase will reveal a misreading due tt) fat tins such as wt)rd'Order dif- 
ferent es. gr»mimatical ditfererucs tn votal jiary words tlial are false 
tognates atrtjss ihe language (i t- . appear to have the same meaning 
but tlo nnl, Ukvlthntui m Spanish arul hhtuu in English). 
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Beyond the Six Principles 



Other Areas of Concern for Ev^iluation 



The presentation of the six principles and accompanying situations 
was purpovly slanted toward evaluation of language-related, cla^s- 
rtxim-centered issues in bilingual education. The main question that 
prompted the iifcntification of such principles was 'What makes bi- 
lingual education evaluation different from other kinds of schcx)! 
evaluation?" And the answer seemed to be The use of two languages 
as vehicles for instruction." Thus, i! is no coincidence that all the prin- 
ciple^i deaf in some way with language-^retated evaluation. This is not 
to say that evaluation of math, scxial studies, science, and so forth, is 
of less impt>rtance. but simply that evaluation in tht*se areas has a 
more solid base of experience to draw from, whereas bilingual educa- 
tion evaluation is a relatively recent phenomenon. Only in the past 
several years has the field of bilingual evaluation l>ecn able to boast an 
array of assessment instruments (see. for example, Disseminalion 
Center for Bilingual Bicultural Education, 1<'75; Silverman et al., 1076. 
Plelcher et al., W78), 

The "newest " aspect of evaluation in the subject areas may. in fact, 
be that subject-matter tests now have to (or should) appear in at least 
two languages. But here again, the problem is primarily one of Im- 
Xfu\^f, Is the minority language test a translatit^n from the English ver- 
sion? If so, is it a meaningful translation, given the stx icKultural con- 
text of the target-language group? If the test is not a translation and 
was not meant to be, then are the English and minority language tests 
comparable in scope and difficulty? There is a host of other questions 
that could also be posed. 

There are areas of concern for evaluation of bilingual education 
programs that are not taken up in the principles discussed pre- 
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viousiy. One concern is that of setting the school in its context. 
Mention of context was made only in passing, with regard to Principle 
1. the program model, type, and design. Various aspects of school 
context could l>e described and evaluated, such as the ethnic composi- 
tion of the school staff, the locus of the program within the school 
structure, the degree of institutionalization of the program, and the 
extent of the program's dependency upon federal funds as opposed to 
local funding sources. The classroom teacher may have some insights 
to share with the ev.^lua^or in these areas. The evaluator is likely to 
obtain some of this information from school administrators, including 
the director of the bilingual program, if there is one. 

Another concern is the cultural background of the students, given 
their particular ethnic affiliations. The role of culture in bilingual 
education is a broad matter and has been treated in detail elsewhere 
(see. for example. Saville-Troike. 1978), Suffice it to say here that an 
evaluator might be concerned with how the student's cultural back- 
ground influences the behavior that ts being assessed. For example, 
when assessing language use patterns, the evaluator would probably 
want to be cognizant of any cultural factors that would influi^nce both 
whom students talk to. in what context, what they talk about, and 
which language or language variety they use. It may be misleading in 
other words, to use some majority-grimp framework for judging con- 
versational skills and appropriate choice of language, when the 
minority culture would have another framework. For example, accept- 
able ways of requesting things in the minority language may appear 
rude in mainstream conversation 

Take the example of a simple request to a student to open the 
door. In the minority language, it may be f^rfectly polite to give the 
direct command. 'Open the door." Depending on the circumstances, 
such a command may sound too rude in English, It may be necessary 
to soften the request through a question, e g , "Could you open the 
dix>r for me?" Also, what appears to be a surprisingly high use of 
English by minority language students in a bilingual program may 
result from the students' desire to please the teacher by using the 
teachers native language, which in this case turns out to be English. 
And if the teacher is a native speaker of the minority language, the 
student may likewise use the minority language more Such student 
accommodation of the teacher's native language has actually^ been 
seen to take place (see, for example. Bruck. Shult/, and Rodrfgue/- 
Brown, 1979), 

Culture alsi-^ proves to be a crucial element in any formal testing 
situation. As Saville-Troike (1978) points out, 'Testing is itself a social 
event . . . Evaluation instruments can never be considered cultui *Ily 
neutral, no matter how objective' their format" (p. 49), We could, for 
example, ask whether the attitudinal measure referred to under 
Principle 1 is perhaps more culturally specific to the Anglo culture in 
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the IMited States. For exampit, fs it ailtuiaily ^s^cific to conceptualize 
attitu<fes In tenra erf sc^ sudi ^ "self -^r^atKe"? Or even If siwii 
scales might have valkiity in <Hher culti^es^ wouM the scales be com- 
posed erf similar items, wonfed in similar ways? Iw exmnpk in 
jOKrt^ culture {>erhaps self-acceptamre b n<rt so inner-<lirected a con* 
cept as in the United States. In am^her culture, accepting one^lf may 
refer first and fc^^most to being a loyal member ctf the ccmmunity. 
Thw, m item assessing self-^xeptaiKe in tfwt culture might investi- 
gate tf« studciit's sense d commitment to the community. Hie item 
would also most likely be phr^i^ in language ccmstet^t with the 
culture, i.e,, in terms of commitment to the community. 

Also, there is more to be said about the academic characteristics of 
the students t>cing evaluated. The emphasis in the disct^slon of ftin- 
ciple 2, student char^terbtics, is on language ard on tl^ relitim- 
ship between language and cognition. The students' ethnicity and the 
history of their involvement in bilingual schooling are abo touched 
on. However, much more than this goes into a student s academic 
profile. 

How well has the student performed in the content areas up until 
now? How well is the stiKlent performing at this point? For instance; 
the student may have completed two years of schooling in some ollwr 
country and may have been weak in math there. Or the student may 
have been in a bilingual program from the start and may have had dif- 
ficulties grasping science ccwKepts, even when they were presented 
exclusively in the student's native tongue. It may be that the diffi- 
culties are due largely to the nature of the instruction. 

For example, the student and the teacheris) may not have gotten 
along well together the curriculum materials may m>t have motivated 
the student adequately, the way the two languages were us^ in the 
classroom may not have been appropriate, and itie like. In the p^t, i^ 
was felt that bilingual chiUren's difficulties in school were likely to be 
a result of limited intellectual ability. Work over the last decade has 
shown that such assumptions were ill founded. For example, DeAvila 
develq>ed a Piagetian-t^ased measure of cognitive development con- 
sisting of five subscaies for each of two levels {K-3, grades 4^) 
(DeAvila and Havassy, 1974; DeAvila and Duncan, 197fib). He then 
tested it with over 6,000 Mexican American children, showing ttiat 
their perfonnance was similar to that of Anglos, He found differences, 
however. In school -related achievement. He concluded that these dif- 
ferences were due to lingufetic and sociocultural biases inherent in 
most of the currently used educational approaches {DeAvila and 
Duncan, 1979). 

Anotfwr factor which was only partially touched on that of 
teacher variables. With respect to Principle 3, teacher and student 
language use patterns, it is suggested that teachers may consciously or 
unconsciously mark the minority language as less prestigious. There 
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are, <rf ccmrse, a number erf ortw teacher factois, such as texiMT pcr- 
sondity, the extent and nature of tether training, the teacher s pro- 
fkiatcy fai t}« two Imgua^ t>^ te^K^ieKs expertence teachfa^ 
th«oi;gJti !he two langu^^, the teadier's airility to provide a learning 
dialkr^ that is compatiUe with the leamii^ pcHential (rf Ihn indhri- 
dual stuiknt the teacher's ability to le^ dtscitssims in dass so as to 
encourage stinlent teaming. A comprehensive listing of tcsdm f^Um 
relating ^)edfically to bilingual instruction has af^ared ebewhere 
(Center for Af^ied Ui^btics, 1974X as well as extensive sugges- 
tions for what teacher-preparation prc^ams in bilingual education at 
the undergraduate and graduate leveb mi^t consist erf (Acosta and 
Blanco, 1978). 

There is tiw prd^lem, however, that teacher variables are not easy 
to evaluate objectively uncfer any circun^tances. A Unguage-ise 
variable, such as the teacher's choice of language for a partkular 
activity or moment within an activity, may be e«ier to assess than 
vari^>ies concerning how effective the teiKiier's system of discipline is 
or how affirmatively the teacl^ respomb to students' attempts to 
exfHess themselves. Even if such tether variables arc assessed ef- 
fectively, bodes like those by Dunkin and Biddle (1974) and Brof^y 
awl Good (1974) seem to have dbpelled the notion that teaching 
patterns such as limited "teacha- talk" (Flanders, 1970) will produce 
more benefkial results than extensive teacher talk. Although a 
theoretical model might suggest that teachers should not talk too 
mucK Dunkin ami Buldle (1974) concluded from a review of various 
studies that there was no relationship between the extent of teacher 
talk and student achievement or attitudes. The study also found 
among other things, that accepting pupils' ideas, praising students, and 
asking frequent questions did not necessarily lead to greater achieve- 
ment on the part of the students. 

Even if we could somehow accurately describe a numt^r of char- 
acteristics of a particular teacher or teachers in a bilingual program, 
how couU we ensure that teachers in a control or comparison group 
(if there is one) display similar characteristics, for comparison pur- 
poses?, We usually canned. Hence, we have the phenomenon, as found 
in the Redwood City study, of comparison-g^oup teachers leaving 
many more years of experience in teaching English reading than bi- 
lingual-program teachers (Cohen, 1975b). Tucker and Cziko (1978) 
suggest ih^k the best that one can do is to make sure that experimental 
and control class teachers are equally qualified. For this, we could turn 
to the list of qualifications for bilingual teacher certification developed 
by the Center for Applied Linguistics (1974). 

The six principles also do not speak to the issue of parent and com- 
munity involvement in the prc^am. Actually, there is a growing con- 
cern regarding the role of parents and of tlw community at large in the 
workings of a bilingual program. For example, aie parents simply a 
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group to be pUcated or infortned, or are they a body that h to impose 
checks ami balam^, or even help to introduce change (Rodriguez, 
1^)7 And at what st^^ of (no^p^am development {i.e.« planning, 
implementation, or evaluation) are parents to be involved (Cruz, 
I979n 

Although time-consuming to document, characteristics of the local 
community can {Ht>vide the evaluator invaluable information to aid in 
interi^eting results. In reality, very little research Imn cc^ucted 
regarding, for example, bilingual language presidency and language 
use patterns of parents in the community, parental sun>ort for tn- 
lingual schooling, or parental knowledge about bilingual programs 
(Cohea 1979). Knowledge alxMit the language prcSiciency and Un- 
guage use patterns of parents erf children in bilingual programs can 
contribute to the construction and selecticm of bilingual cuniculum 
materials appropriate to t)^ language t>^kground of the students. 
Such knowledge could for example, help avert the development c( 
materiab that are too demanding in one language and not demanding 
enough in the other. Longitudinal research on language use patterns of 
families with children in bilingual progran^ can help inform program 
administratof^ about whether prc^am language goals are being met 
(e.g., whether the minority language is l>eing maintained, if this is a 
goal; see Cdhen, 1975b, clupter 9). Sometimes, information about the 
local community can be gleaned from school visits with parents, from 
home vfeils, from school trips, and in other ways. 

Regarding parental involvement in the bilingual prc^ram, it is not 
a given that parents of children in a bilingual program are supportive 
of the program. Yet initial misgivings about bilingual schooling may 
give way to more positive feelings as parents see the results of the 
program. If parents are to make genuine choices about the bilingual 
schooling of their children, then they may need to be involved in pro- 
gram specifics at the three levels of planning, implementation, and 
evaluation, rather than simply endorse an al^straction, namely 'bi- 
lingual education/' It is possible that teachers can help parents play this 
sort of role and can help evaluators understand what the parental and 
community roles actually are in a given case, as well as what they 
could he. 

There is still a need to improve questioning procedures in order to 
tap parental knowledge and opinions about bilingual schooling. Per- 
haps teachers can help obtain information beyond pat answers to 
interview questions. What is needed is an informal, natural environ- 
ment in which honest comments would emerge. Sometimes evalua- 
tors can effect this, and sometimes they cannot. Teachers may be able 
to help. 
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This monograph has concentrated on how teachers can assist evalua- 
tors because clearly the teachers themselves do not have much lime in 
their schedules for conducting evaluation, particularly formally. But 
teachers may have time for more informal evaluation, such as sug- 
gested in the vignettes. In fact some of the descriptive information 
about the classroom may be relatively inaccessible to anyone but the 
teacher since it reflects cumulative insights gleaned from repeated en- 
counters with students on a daily basts. 

Let us take a look now in summary at some of the activities which 
have been suggested for teachers: 

t Gathering descriptive facts about how the bilingual schooling 
model type, and design actually function on a daily basis 

Z, Providing data on students' language proficiency and on their 
cognitive f'joctioning (ie., their grasp of corKepts) in both their 
native language and in their second language 

3. Describing the teaching methoik in the prt^gram (the approaches 
that are used on a daily baste), the teaching materials and how 
they are used, and any changes that occur in methods ar^i 
materials over time 

4. Gathering data, possibly tape-recorded, on teacher and student 
language use pattens 

5. Reporting on student facility with speech acts (like requesting, 
suggesting, and apologizing) in both languages 



6. 



Paying special attention to reading development in both lan- 
guages, particularly among students with reading difficulties^ so 



that tl^ teacher can provide insights to evaluators about what the 
reading test results mean in these aises. 

It is a tall order to ask teachets to ackl more activities to t^ir ai- 
re^y busy schedules, but it may just be tlut activities such as tho^ 
listed here will be welcome additions (assuming teachers are not al- 
ready engaged in such activities, which ih^y well may l>e). Not only 
would teadwrs inaei^ the information for evaluators about what h 
being evaluated but at the same time they may also bring their own 
conceptions of the program into sharper focus. 
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